Accelerated Primal-Dual Proximal Block Coordinate Updating Methods for Constrained Convex Optimization

Block Coordinate Update (BCU) methods enjoy low per-update computational complexitybecause every time only one or a few block variables would need to be updated among possiblya large number of blocks. They are also easily parallelized and thus have been particularlypopular for solving problems involving large-scale dataset and/or variables. In this paper, wepropose a primal-dual BCU method for solving linearly constrained convex program in multi-block variables. The method is an accelerated version of a primal-dual algorithm proposed by theauthors, which applies randomization in selecting block variables to update and establishes anO(1/t) convergence rate under convexity assumption. We show that the rate can be acceleratedto O(1/t) if the objective is strongly convex. In addition, if one block variable is independent ofthe others in the objective, we then show that the algorithm can be modified to achieve a linearrate of convergence. The numerical experiments show that the accelerated method performsstably with a single set of parameters while the original method needs to tune the parametersfor different datasets in order to achieve a comparable level of performance.